<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">


<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    
    <title>Spark Progress Monitor &mdash; PixieDust Documentation</title>
    
    <link rel="stylesheet" href="_static/better.css" type="text/css" />
    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
    <link rel="stylesheet" href="_static/custom.css" type="text/css" />
    
    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
        URL_ROOT:    './',
        VERSION:     '1.0',
        COLLAPSE_INDEX: false,
        FILE_SUFFIX: '.html',
        HAS_SOURCE:  true
      };
    </script>
    <script type="text/javascript" src="_static/jquery.js"></script>
    <script type="text/javascript" src="_static/underscore.js"></script>
    <script type="text/javascript" src="_static/doctools.js"></script>
    <link rel="shortcut icon" href="_static/pd_icon.ico"/>
    <link rel="top" title="PixieDust Documentation" href="index.html" />
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
  </head>
  <body role="document">
    <header id="pageheader"><h1><a href="index.html ">
        PixieDust Documentation
    </a></h1></header>  

    <div class="document">
      <div class="documentwrapper">
        <div class="bodywrapper">
          <div class="body" role="main">
            
  <div class="section" id="spark-progress-monitor">
<h1>Spark Progress Monitor<a class="headerlink" href="#spark-progress-monitor" title="Permalink to this headline">¶</a></h1>
<p>When you run a Spark job within a Python or Scala notebook, you can&#8217;t see how the job is progressing. Wouldn&#8217;t it be nice to have a progress bar that shows you how your job is running and provides details? PixieDust includes a Spark Progress Monitor bar that lets you track the status of your Spark job. No more waiting in the dark. Notebook users can now see how a cell&#8217;s code is running behind the scenes.</p>
<p>To invoke the Spark progress monitor, run the following command in your Python notebook:</p>
<blockquote>
<div><div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">pixiedust</span><span class="o">.</span><span class="n">enableJobMonitor</span><span class="p">()</span>
</pre></div>
</div>
</div></blockquote>
<p>Doing so invokes the progress monitor for the session. Each time you run a Spark job after that, the monitor appears.</p>
<img alt="_images/sparkmonitor.png" src="_images/sparkmonitor.png" />
<p>To see details, click the blue &gt; carat:</p>
<img alt="_images/monitor_carat.png" src="_images/monitor_carat.png" />
<p>and PixieDust shows you more information about the job:</p>
<img alt="_images/monitor_details.png" src="_images/monitor_details.png" />
</div>


          </div>
        </div>
      </div>
      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
        <div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li class="toctree-l1"><a class="reference internal" href="use.html">Use PixieDust</a></li>
<li class="toctree-l1"><a class="reference internal" href="develop.html">Develop for PixieDust</a></li>
</ul>

<div id="searchbox" style="display: none" role="search">
  <h3>Quick search</h3>
    <form class="search" action="search.html" method="get">
      <input type="text" name="q" />
      <input type="submit" value="Go" />
      <input type="hidden" name="check_keywords" value="yes" />
      <input type="hidden" name="area" value="default" />
    </form>
    <p class="searchtip" style="font-size: 90%">
    Enter search terms or a module, class or function name.
    </p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
        </div>
      </div>
      <div class="clearer"></div>
    </div>
  <footer id="pagefooter">&copy; 2017, IBM.
      Created using <a href="http://sphinx-doc.org/">Sphinx</a>
      1.3.5.

  </footer>

  
  </body>
</html>